3D Face Reconstruction Under Imperfect Tracking Circumstances Using Shape Model Constraints

We propose a factorization structure from motion (SfM) framework which employs 3D active shape constraints for a 3D face model application. Two types of shape model, individual shape models and a generic model, are used to approximate non-linear manifold variation. When the 3D shape models are trained, they help the SfM algorithm to reconstruct the 3D face structure under noisy observation (tracking) circumstances. By minimizing two sets of errors, the reconstruction error generated by the linear transform of the shape models and projection error obtained by re-projecting the 3D shape to 2D positions, the 3D face shapes can be recovered optimally. Experimental results show that this algorithm accurately reconstructs the 3D shape of familiar and non-familiar faces from video sequences under circumstances of imperfect face tracking or noisy observations.